[J.Atmos 



..Sci.68,1507-1512(2011) J^opyright July 2011 AMS| 

Maximizing the statistical diversity of an ensemble of bred vectors 
by using the geometric norm 



Diego Pazo,' Miguel A. Rodriguez,' and Juan M. Lopez' 

' Instituto de Fi'sica de Cantabria (IFCA), CSIC-Universidad de Cantabria, E-39005 Santander, Spain 

We show that the choice of the norm has a great impact on the construction of ensembles of bred vectors. 
The geometric norm maximizes (in comparison with other norms like the Euclidean one) the statistical diversity 
of the ensemble while, at the same time, enhances the growth rate of the bred vector and its projection on the 
linearly most unstable direction, i.e. the Lyapunov vector. The geometric norm is also optimal in providing the 
least fluctuating ensemble dimension among all the spectrum of g-norms studied. We exemplify our results with 
numerical integrations of a toy model of the atmosphere (the Lorenz-96 model), but our findings are expected 
to be generic for spatially extended chaotic systems. 



I. INTRODUCTION 

The 'breeding method' is a well-established and compu- 
tationally inexpensive procedure for generating perturbations 
for ensemble integrations f2T,'2?]. Bred vectors (BVs) are 
finite perturbations periodically rescaled to a certain magni- 
tude that have been prominentlyused in probabilistic weather 
forecasting with ensembles f^io]. The breeding method and 
variants of it are applied in operative ensemble forecast sys- 
tems, such as National Centers for Environmental Predictions 
(NCEP, USA), see e.g. [24]. Moreover, breeding continues 
to be a popular tool to study the predictability of a variety of 
systems such as the baroclinic rotating annulus Il26tl or the at- 
mosphere of Mars l,14il . 

Different initial BV perturbations all generally tend to be- 
come aligned with the fastest growing modes. If different BVs 
were globally quasi-orthogonal to each other |(22|, one might 
expect they would automatically provide a good sample of the 
different dominant growing error directions, without the need 
for additional computation. A closer inspection reveals that 
the BV perturbations are often locally rather similar in shape, 
differing only in sign and amplitude f?, '22*]. In fact, a major 
modification of the BV implementation at NCEP has recently 
been implemented by replacing the BVs given by the ensem- 
ble forecast with some 'ensemble transform' that orthogonal- 
izes the ensemble with respect to the metric defined by the 
inverse covariance matrix ||3L |23| - |25[1 . Other metrics can be 
used and lead to different ensembles of BVs U0.1 . Orthog- 
onalization with respect to a given metric generally enhances 
the statistical diversity of the ensemble by making the BV per- 
turbations globally more dissimilar |2, 10]. 

In this paper we show how the ensemble diversity can be 
enhanced by using the geometric norm with no further trans- 
forms or orthogonalizations needed. We first show that the 
BVs dynamics and the statistical properties of the ensemble 
strongly depend on the norm definition used to construct them. 
So far Euclidean-type norms are widely used in applications. 
However, our results demonstrate that, among a spectrum of 
studied norms, the geometric norm is the most convenient be- 
cause it provides a greater statistical diversity of the ensemble, 
while it enhances the projection of the ensemble as a whole on 
the most unstable direction. With other norm choices, like the 



standard Euclidean one, a good projection on the leading Lya- 
punov vector (LV) is always associated with the collapse of 
all the BVs, i.e. the complete loss of the ensemble diversity. 



II. THE MODEL 

We illustrate our study with numerical integrations of the 
well-known Lorenz-96 model ifTTIl that has been used by var- 
ious authors as a low order testbed for atmospheric prediction 
and assimilation studies 1 1, 13]. This model is defined by the 
set of variables {u{x, t)}x=i l and evolves according to 

^"^^'^^ = -u{x-l,t)[u{x-2,t)-u{x + l,t)] 

- u{x,t)+F, with x = l,...,L. (1) 

with periodic boundary conditions in the discrete spatial vari- 
able X. Hereafter we adopt a system size of L = 128 and a 
forcing constant F = 8. For these values the system exhibits 
well developed chaos [12] . 

A good description of the chaotic dynamics can be achieved 
by understanding the behavior of initial infinitesimal perturba- 
tions, which are governed by the 'tangent linear model': 

dSu(x, t) ^ , , r / s , 

= -5u{x - 1, t) [uix -2,t)- u(x + 1, t)] 

at 

-u{x- l,t) [Su{x - 2,t) - 6u{x + 1,^)] - 6u{x,t). (2) 

After some transient any infinitesimal perturbation 6u{x,0) 
becomes permanently aligned along the most unstable direc- 
tion. This direction defines, disregarding an arbitrary nonzero 
constant factor, the leading LV, and hereafter denoted g(t) = 
{g{x,t)}^=i,,,„L. 

Obtaining the tangent linear (and adjoint) models can be 
however extremely difficult in operative weather models and 
one has to resort to analyzing finite perturbations, which are 
evolved with the full nonlinear model. This is for instance the 
situation at NCEP, where ensembles of BVs are used. 
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III. BRED VECTORS 

BVs are finite perturbations obtained after periodic rescal- 
ing, say at times i,„ = niT (m G Z+). A control trajec- 
tory u and a perturbed one u', are simultaneously integrated 
[via Eq. ([U] and at the scheduled times the difference between 
them is calculated 



and rescaled to a given amplitude e, obtaining the BV 

Au(UO 

^^'"^ ||Au(i„)|| 

This BV is then used to redefine the perturbed system: 

u'{t+) = u(t,„) +b(t,„)- 



(3) 



(4) 



(5) 



with t+ = limi^^o tm + v. The perturbed u' and control u 
states are then evolved in time according to the model equa- 
tions, Eq. ([TJ, until the next scheduled rescaling. At the next 
scheduled time t„i+i the breeding cycle, Eqs. (O-©, is re- 
peated. After several breeding cycles, the perturbations gen- 
erated by this procedure acquire a large growth rate, which 
makes them suitable for ensemble forecasting. Usually a set 
of BVs is evolved from different initial random perturbations 
and this constitutes the ensemble. Ideally a good ensemble of 
BVs should span the most unstable directions in phase space 
well enough to capture the main instabilities. 

There are three basic ingredients in the definition of the B V: 
(i) the rescaling interval T, (ii) the perturbation amplitude e, 
and (iii) the choice of the norm || • || used in Eq. (|4]i. 

The rescaling interval T has a negligible influence in the 
results as long as it remains small — say, smaller than the dou- 
bling time, which is on the order of 0.4 time units (t.u.) for 
the Lorenz-96 model. We have used T = 0.1 t.u., which cor- 
responds to 1 /2 day in the time scale assumed by [11]. 

The perturbation amplitude e controls the "finiteness" of 
the perturbations; a sufficiently small e makes the perturbation 
quasi-infinitesimal, and in the limit e the BV perfectly 
aligns with the leading LV of the system. 

However, very little is known about the effect of the norm 
choice on the properties of the resulting ensemble and we dis- 
cuss this issue in detail in the incoming sections. 



IV. CHOICE OF A NORM 

The choice of the norm is probably the more obscure el- 
ement determining the BVs' nature. BVs have often been 
claimed to be insensitive to the choice of norm ^ ^ . How- 
ever, this belief is not actually based on any rigorous argu- 
ment. Here we show that the effect of changing the norm 
type has a dramatic impact on BVs. We will show that differ- 
ent norms lead to different ensemble properties and it is not a 
mere change of the 'ruler' or metrics. There are intrinsic and 
genuine effects on the statistics of the BVs for each particular 
norm type. 



Intuitively, for a homogeneous system like the Lorenz-96 
model, any definition for the norm one wants to use should 
be homogeneous in the sense that it weights equally all sites. 
To see why this constraint is relevant let us consider a particu- 
larly illustrative example. Think of a norm arising from some 
scalar product ||b|p = (b, b) = b^Mb with a very "un- 
balanced" metric matrix M, e.g. M ~ diag(100, 1,1,..., 1). 
This choice would result in very dissimilar BVs depending 
if the site a; = 1 is more or less unstable at a given time. 
For a given e, at some times the vector dynamics could be 
infinitesimal-like while at other moments it would be clearly 
finite. For spatially homogeneous systems, it is reasonable to 
restrict ourselves to "homogeneous" norms that produce a BV 
that is statistically equivalent up to a high degree at different 
times and we do so in our study. 

In this work we compare the performance of g-norms, 
which are defined as 



lAull 



x = l 



1/9 



(6) 



Note that for q = 2 the norm is an energy-like norm, analo- 
gous to those used in atmospheric models. In the limit q — > oo 
the q-norm becomes the supremum norm: 

||Au||,^oo = sup{|Aw(x,t)|}a;=i,...,L. (7) 



Moreover, the geometric mean is obtained in the Umit(l2 



iAuiio=ni^"(^'^)i 



1/L 



(8) 



The use of t he g eometric norm yields the so-called logarith- 
mic BVs ll7l ll7l - l20ll . For clarity of presentation we will add 
the subscript q to the notation (for the BV, hq, and the am- 
plitude. Eg) to emphasize which g-norm is being used. For 
all (/-norms the BVs look like very similar to the naked eye 
is strongly localized in space, and are strongly localized in 
space for small Eg. (see for instance [7] and [18] for typical 
snapshots of BVs with q = and q — 2). 

Along this paper we shall be considering an ensemble of 
/c = 10 BV members, {bg^'l^^i . All members of the 
ensemble are simultaneously rescaled, and they are initiated 
with independent random initial conditions, which is expected 
to result in some degree of diversity in the ensemble of BVs 

m. 



V. RESULTS 

We define ensemble diversity as the degree of linear in- 
dependence or transversality among the ensemble members. 
Diversity can be quantified by calculating the 'ensemble di- 
mension', which measures the effective dimension of the sub- 
space spanned by the ensemble. The higher the ensemble di- 
mension the greater the statistical diversity. Higher ensemble 
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FIG. 1 : Average ensemble dimension as a function of the amplitude 
10 BVs, with different g-norms. 



dimension would imply larger dissimilarities among the en- 
semble members. Since BV perturbations tend to align with 
the fastest growing modes, a greater diversity indicates that 
the ensemble is able to actually sample not only the main LV 
but also other, less unstable, directions. 

Our goal here is to show, by means of several numerical 
calculations with a simple model, that the 0-norm is more con- 
venient than other norms for constructing ensembles of BVs 
as far as ensemble diversity enhancement is concerned. We 
arrive to this conclusion measuring the ensemble dimension 
and its temporal fluctuations, the average growth rate, and the 
alignment of the ensemble members with the main LV. 



A. Ensemble dimension 

In this subsection we will analyze the statistical diversity 
in an ensemble of k BVs. Clearly, for all q values, in the 
limit Eq ^ all BVs become aligned with the leading LV and 
there is no diversity in the ensemble (other than a global sign 
for the orientation of the vectors). If Sq is finite some degree 
of transversality between ensemble members can be expected, 
and to measure this diversity of the ensemble we resort to the 
so-called ensemble dimension |4i] . 

The ensemble dimension ifTsIl or BV dimension lfT6|] was 
proposed as a way to account for the number of effective 
degrees of freedom that explains most of the total ensemble 
variance (in the spirit of principal component analysis), see 
e.g. I'T] and references therein. To compute the ensemble di- 
mension at a given time one computes the k x k covariance 
matrix C with elements: 



L\\hi:^\\2\\hi'^\\2' 



(9) 



tor ^4'\b^^'^^ = J2.bi"\x,t)b'!j'\x,t). If we denote by 

{lJ'i{t)}i=i....,k the set of eigenvalues of C, the ensemble di- 
mension is: 



(10) 



where the standard scalar product is used in the numera- 



where the denominator (^ ■ fii) equals k due to the normaliz- 
ing terms in the denominator of The statistic ( fTOl ) typically 
returns a real number between two limit values: Den = k (if 
all vectors are orthogonal) and Den = 1 (if all vectors are 
aligned). Therefore Dcn{t) measures the instantaneous de- 
gree of "transversality" of the ensemble. 

Figure [T] depicts the results of the time-average ensemble 
dimension (Den) for different g-norms. Depending on the 
value of q the amplitude Sq is varied in a different range. 
The largest value of Sq in each data set corresponds (approx- 
imately) to the value of the average distance (for the cor- 
responding q-norm) between independent realizations of the 
model (i.e. random climatological values). In applications Eg 
is much smaller than this value (typically of the order of the 
analysis eiTor). In the small Eg region of the plots, (Den) be- 
comes equal to 1 below a particular value of Eg, though for the 
0-norm the convergence to 1 appears to be much less abrupt. 



B. Fluctuations of the ensemble dimension 

Figure [T] shows that all g-norms allow to obtain ensembles 
with a certain (Den) after tuning Eq to a particular value. How- 
ever the ensemble dimension is a time-fluctuating quantity and 
one should wish to minimize its fluctuations. Of course some 
degree of fluctuations is unavoidable due to (i) finiteness of 
the ensemble and (ii) intrinsic fluctuations in the state of the 
system (which progressively average out for large enough sys- 
tems). 

We characterize the fluctuations of Den by means of the 

/ 2\^/^ 

Standard deviation cr = ([Dcn{t) — {Den)] ) where the 
brackets denote a temporal average. The results are depicted 
in Fig. 12] where we plot the relative fluctuations of the en- 
semble dimension versus (Den) to better compare different 
g-norms. One can readily see that the 0-norm produces the 
ensemble with the smallest fluctuations for most (I?cn) val- 
ues. 



C. Alignment with the main Lyapunov vector 

Ideally (i.e. disregarding Umitations by numerical accu- 
racy) the BVs become perfectly aligned with the main LV as 
Eq —i' 0. To determine quantitatively the degree of alignment 
with the LV, g(i), we have measured the instantaneous angle 
between each BV of the ensemble, b^*'' (t), and g(t) at breed- 
ing times t = tm SiS customary in a L-dimensional Euclidean 
space ll28ll : 



0«(i) = /(g(< = i„),b«(t = i„)) 



(11) 
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FIG. 2: Relative fluctuations of the ensemble dimension. 
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FIG. 3: Average angle between the BVs and the main LV. 



The ensemble and time average angle ((/>) is shown in Fig. [3] 
and demonstrates that the logarithmic BVs (q = 0) are able 
to achieve a considerable degree of alignment with the LV 
on average, while retaining some degree of diversity. One 
clearly sees that BVs constructed with norms q > become 
strongly aligned among themselves while still keep a high de- 
gree of transversality with the main LV, as reflected by the 
high average angle of the ensembles {{(f)) > 7r/4) in Fig.|3]for 
(Den) — 1- In contrast, the 'logarithmic ensemble' (q — 0) 
exhibits a lower angle with the main LV, even if the statistical 
diversity is high. We claim that the higher diversity and lower 
(</)) exhibited by the ensemble of logarithmic BVs {q = 0), as 
compared with the ensembles with q > 0, indicates that this 
ensemble is spanning a sub-space formed by a naiTow hyper- 
cone around the main LV, while ensembles with q > tend 
to lie in a lower dimension subspace that is more transverse to 
the LV. 



D. Average growth rate 

Also the growth rate of the ensemble members can be used 
to compare with that of the main LV, reflecting again the dif- 
ferent behavior for different norm choices. The average expo- 
nential growth rate of the bred vectors is 



T 



In 



\Au{tra+T)\\ 
I|b9(*m)||2 



(12) 



Notice that, for the sake of clarity we are using the same norm 
type (q — 2) to measure the exponential growth rate in all 
cases (nevertheless due to the long averaging the norm type is 
irrelevant). 

Figure |4] shows the dependence of A on the ensemble di- 
mension. One can see that the logarithmic BVs {q — 0) ex- 
hibit the largest amplification rate for a given ensemble dimen- 
sion, which is in agreement with the results discussed in the 
preceding subsections showing that the logarithmic ensemble 
(q — 0), among all ensemble choices, exhibited the greatest 
projection on the LV. Conversely, given an exponential growth 
rate A, using the 0-norm will result in the most diverse ensem- 
ble. 



VI. CONCLUSIONS 

We have studied the effect of different norms on the con- 
struction of ensembles of BVs. The geometric (q — 0) norm 
outperforms other norms (like the Euclidean one, q — 2) 
for constructing ensembles of BVs in spatially extended sys- 
tems. The enhancement of performance (in terms of root- 
mean square eiTor, ensemble spread, and calibration time) of 
ensembles of logarithmic {q = 0) bred vectors with respect to 
standard "Euclidean" (q ~ 2) bred vectors was akeady uncov- 
ered by li 1 8i1 . In the present work we give a rationale behind 
those results. We show that an ensemble of logarithmic BVs 
(obtained with the 0-norm) exhibits greater diversity — larger 
ensemble dimension — while its members are more strongly 
projected on the leading LV and have growth rates that rapidly 
approach the leading Lyapunov exponent. In comparison, en- 
sembles based on BVs with (7 > perform rather poorly. They 
tend to collapse in one single direction (i.e., (Den) — 1) very 
abruptly as the B V amplitude is diminished and, even when all 
the statistical diversity is lost, they remain rather transverse to 
the leading LV as demonstrated by the angle with the main LV 
shown in Fig. [3] Moreover, the geometric norm also leads to 
the least fluctuating ensemble dimension among all the possi- 
ble g-norms. 

In the view of these results two prominent questions 
remain open. On the one hand, it would be very interesting 
to evaluate the performance of 0-norm BVs in real appli- 
cations. The study by ifisll of 0-norm BVs already showed 
promissing, albeit preliminary, results. Clearly, more research 
is needed in this direction. On the other hand, there is the 
problem of analyzing the potential advantages of ensemble 
Kalman filters based on 0-norm BVs. Our results show 
that logarithmic BVs have very nice properties regarding 
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statistical diversity, growth rates, and projection onto the 
main LV. Therefore, a natural question that arises is: to what 
extent can these features translate into a better performance 
of ensemble Kalman filtering methods?. We beUeve our re- 
sults may serve as a basis for future research along these lines. 




(^en) 

FIG. 4: (a) Average growth rate A as a function of the average ensem- 
ble dimension. The dotted line indicates the value of the Lyapunov 
exponent, (b) Zoom of panel (a). 
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